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DATA COMMUmCATlONS 

This invention relates to data coinmumcations, and in particular to the 
communication of muUimedia documents from multimedia servers to 
5 multimedia clients. 

The Internet currently provides users with a number of different 
services. One of such services is the Worldwide Web, or "the Web", which has 
now become relatively weli-known<ind well- used by fixed terminal subscribers. 
To access the Web, a user instals a "browser" application on a terminal, 
1 0 which acts as a client interfacing with a network of distributed Web servers via 
the Internet. 

In order to access a page on the Web, a user specifies a universal 
resource locator (URL), and transmits a page request via a data link to the 
Internet. In the Internet, the request is routed to the appropriate Web server as 

X 5 identified in the URL. In return, the Web ser\'er transmits the pages back to the 
requesting browser. The communications protocol used within the Internet is 
TC/IP during this dialogue. 

Web pages are currently formatted using the HyperTexl markup 
language (HTML). A Web browser, on receipt of a HTML document, 

20 interprets the HTML commands in order to format the text correctly. 

HyperText links may be included in the document which, when presented to the 
user, may be selected in order to request a further Web page (a HTML link 
references a further URL with which the browser retrieves the further page). 
Web pages ciirretitly contain not only text and hyperlinks, but also image, audio 

2 5 and./or video files (hence, Web pages are referred to as "multimedia" 

documents). 

Although the textual data in the Web page is generally relatively data- 
compact, it is recognised that graphics files, audio files, and in particular video 
files, contain relatively large amounts of data which can reduce related 

3 0 download of Web pages considerably. As a result, the downloading of "true" 

multimedia documents including graphics, audio and/or video files can be 
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cumbersome, in particuiar where the data link used into the Internet is of a type 
which is of a relatively low data rate. 

Hie speed of download of muhimedia Web pages is particularly 
problematic in the case of mobile data Hnks such as a cellular radio data link, in 
5 which the radio data transfer is constrained by the relatively low bandwidth of 
the radio interface link. 

Therefore, although it is cuirently possible to access the Web at a mobile 
communications terminal, the option is currently not widely taken advantage of 
and, when taking advantage of, the data transferred is generally limited to 
1 0 textual data to achieve reasonable download speeds. 

Methods are known whereby the amount of data transfer involved in 
accessing the Web arc reduced. In particular, many browsers support a caching 
facility whereby a Weh page, oace downloaded, is temporarily stored locally on 
the client terminal. Wlien the Web page is next requested, the browser 
15 recognises, by the URL in the request corresponding with the URL of the 
cached document, that the document is currently held locally, and retrieves the 
iocaliy-heldpage in preference to re-downloading the page from the Web. The 
browser may transmit an update check to the Web server in question in order to 
confirm that the locally-stored Web page remains valid, and if not, downloads 
2 Q the updated Web page. 

Caching may be performed on a session-by-session basis, or may be 
performed such that a cached Web page remains stored between sessions. 
However, a browser mairnain.s only a limited non-volatile cache of Web pages, 
and newly-cached Web pages are stored in preference to previously-stored Web 
2 5 pages (which are then deleted) when the cache becomes Ml. 

In accordance with one aspect of the present invention there is provided 
a multimedia client terminal, said terminal comprising: 

a browser for interpreting a multim.edia document received from a 
remote server, said interpreting means comprising: 
30 means for recognising texmal presentation markup lags in said 

document and presenting text to a user in accordance with said markup tags; 
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means for recognising a standard set of document-independent local 
library file markup tags in said document; 

means for storing a set of non-textual local library files corresponding to 
said local library tags; and 
5 means for presenting the contents of one of said local library files to a 

user in response to the recognition of one of said local library file tags in smd 
multiniedia document. 

Rather than needing to download non-textnai files accompanying the 
text of a document, the client terminal is able to receive one or more of a known 
1 0 set of document-independent and data-efFic ient markup tags and access locally- 
stored files which correspond to the tags in the downloaded document. 

The arrangement of the present invention is to be contrasted with the 
arrangement in which the client terminal holds a non-volatile cache {although, 
such a cache may be used in combination with the present invention), in that 
15 caching mechanisms do not utilise document-independent iocai library markup 
tags. Instead a document is cached by selection of & URL, and it is unlikely 
that, when a new multimedia document is downloaded, any parts of the 
document will already be held locally. 

The use of a non-volatile cache is non-systematic, and does not allow a 

2 0 server content developer to design multimedia documents specifically for low 

data-rate communicaiionlinkjs, which is possible with the present invention. 

in addition, a non-volatile cache arrangement does not allow a user of a 
low data-rate receiving terminal to be confident that a new multimedia 
document cim be downloaded at an acceptable speed, which is possible with the 
25 present invention. 

By defining a standard set of document-independent local library file 
tags, the present invention allows a server content developer to include one or 
more of such tags within a document to be placed on a server in the knowledge 
that a user, provided with a client application for recognising the predefined set 

3 0 of iocai library tags and for retrieving appropriate iocaliy-stored flies, will be 
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abie to download the document rslatively quickly and receive each of the 
intended multimedia parts as intended. 

The present invention is of particular utility in relation to mobile 
communications terminals, 
5 Further aspects of the invention are set out in the appended claims. 

Further features of embodiments of the invention are as foilows : 
1 . The system employs tags to locally stored graphical images, 
drawing primitives, pre-drawn images, image manipulators, audio and video 
files. 

10 2. A first set of tags provides basic two dimensional dra'wing 

fiinctionaHiy, enabling the creation of a range of simple but effective composite 
images by means of combinations of drawing primitives. 

3. The client functionality supports a standard predefined range of 
pre-drawn images that can add further graphical content to a page, over and 

15 above that achieved using the drawing primitives. This enables an image to be 

retrieved from the local repositorj' to be displayed at the specified co-ordinates. 
A scaling factor may be specified to dynamically resize the image and a 
hyperlink can be optionally provided 

4. The client functionality supports the manipuiation of on-screen 
20 bitmap images. 

5. A second set of tags provides basic audio playback capability. 
The system employs tags to stored speech files as well as ttie ability to interpret 
"talking Web pages" through text to speech translation software. 

6. The client fiinctionaiiiy supports a standard predefined range of 
25 audio files enabling a file to be retrieved from tlie local repositoiy, and spoken 

by a speech agent at the client. 

7. The speech agent supports defatilt vocal and facial styles in order 
to allow different genders, accents and languages into the playback. Different 
facial appearances may be selected for the agent- The combination of attributes 

3 0 may be used to create agents with different personalities. 
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S, A third set of tags provides basic video clip playback, capabiiity. 
The system employs tags to stored video files. 

9. The client functionality supports a standard predefined range of 
video files enabling a fiie to be retrieved from the local repository and replayed 

5 at the client. 

10. In one embodiment, tags implementing the new functionality are 
embedded into Web pages, which may also contain HTML, and content 
handlers filter out the tags Impiementing the new functionality before resolving 
them. 

10 11. In another embodiment, references to files implementing the 

new functionalhy are embedded into Web pages, which may also contain 
HTML, as markups (e.g. for graphics resources <imgsrc-"images/text.hgm]">; 
for audio resources <spesrc-"audio/test.hsml>; and for video resources 
<vidsrc= "video/test.hvmr'>). 

15 12. In a further embodiment, pages implementing the new 

ftinctionality are a separate resource, which may be hyperlinked in the 
conresponding Web pages containing HTML (e.g. 
http ://www.test,com/ texthgrni). 

13. The drawing functionality provided by tags to graphics 

2 0 primitives supports Web page navi gation by means of hyperlinks. 

Embodiments of the invention will now be described, by way of 
example only, with reference to the accompanying drawings, wherein: 

Figure 1 shows a schematic block diagram illustrating a document 
retrieval system in accordance with an embodiment of the invention; 
25 Figure 2 illustrates a client'server flinctionaiity in accordance with an 

embodiment of the invention; 

Figure 3 illustrates a predefined set of document-independent graphics 
tags and a set of corresponding graphics elements; 

Figure 4 illustrates an alternative set of graphics elements to be used 

3 0 with the predefined set of graphics tags illustrated in Figure 3; and 
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Figure 5 illustrates a screen display provided in accordance with an 
embodiment of the invention. 

Referring to Figure K a multimedia document retrieval system in 
accordance with an embodiment of the invention includes a mobile client 
5 terminal 2 commimicating via a mobile communications network 4, such as a 
GSM digital mobile communications network, and the Internet 6 with the Web 
server 7. 

The mobile client terminal 2 includes a user interface H, a client 
application 10, a radio communications front end 12 and a radio transceiving 
10 antenna 14. 

The user interface 8 includes a keypad, a display screen and a 
loudspeaker for accepting user input and outputting images and audio signals 
respectively. 

The client application 10 is a customised Web browser, such as a 
IS customised version of ihe Netscape Explorer (Regisiered Trademark) browser, 

and includes standard content handlers such as a text content handler 16 and an 
image content handler 18. 

In addhion, the client application 10 includes, as provided in accordance 
with the present invention, a graphics tag handler 20, an audio tag handler 22 
20 and a video tag handler 24. These content handlers may be components of the 
browser 10 produced in a development language such as Java, or JavaBeans 
(Registered Trademark), Associated with each of the tag handlers are a standard 
graphics librar>' 26, a standard audio library 28 and a standard video library 30, 
containing content files which are document-independent. The graphics library 
25 consists of graphics files GF 1-X. The audio library consists of a number of 
audio files AF 1 -Y, and the video library consists of a number of standard video 
flies VP l-Z, 

The radio commimications front end 12 includes amplifiers, frequency 
selective circuits, etc which are used to perform radio communications via the 
3 0 radio transceiving antenna 1 4. 
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The mobile client terminal 2 may be one of various types of terminal, 
for example a laptop computer provided with radio communications facilities, a 
personal digital assistant (PDA), or a mobile handset provided with Web 
browsing capabilities. 
5 Radio signals transmitted and received by the radio transceiving antenna 

14 of the mobile client terminal 2 are received and transmitted by a cell site 
antenna 32 of the mobile communications network 4, via a radio link 33. Only 
one cell site antenna is illustrated but it will be appreciated that a large number 
of such antennas are provided in the mobile communications network 4, and the 
10 particular serving cell site antenna will depend on the location of the mobile 
client terminal 2 at any particular time. The cell-site antenna interfaces, via a 
base station 34^ to a mobile switching exchange 36 of the mobile 
communications network, which maintains a circuit connection 38 with an 
internet gateway 40. 

15 The Internet gateway in turn interfaces via the Internet 6 to the Web 

server 7, using a TCP/IP connection 42, 

The server 7 holds a set of Web pages 44, each being referenced by 
different URLs. 

The contents of one such Web page 46 are illustrated in Figure 1 . The 
20 Web page 46 contains a textual portion 48, to be handled by the text content 
handler 1 6, an image file portion 50, to be handled by the image content handier 

1 8, one or more of a standard predefined set of document-independent graphics 
tags 52, to be handled by graphics tag handler 20, one or more standard 
predefined document-independent audio tags 54, to be handled by audio tag 
25 handier 22, and one or more of a standard predefined set of document- 
independent video tags 36. Others of the Web pages 44 also contain text 
content and the graphics tags, audio tags and video tags used in the illustrated 
document 46 may be repeated in the remaining Web pages of the set 44 held on 
the server. 

3 0 When the client terminai 2 transmits the appropriate URL request to the 

server 7, via the various data links indicated, the server 7 transmits the Web 
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page back, in HTML format, to the mobile client terminal 2. At the client 
terminal 2, the various content handlers 1 6-24 interpret the HTML to present the 
text and image content 48, 50 of the Web page as indicated by the HTML 
command therein, and the tag handlers recognise the particular graphics, audio 
5 and/or video tags referenced in the document 46, and present the contents of the 
corresponding file in the libraries 26, 28, 30 as indicated by commands 
associated with the respective tags. 

Figure 2 iilustrates functionaiily of a clietit'seiTcrat the content handler 
ievel, in accordance with one embodiment of the invention. In this 
10 embodiment, three different "markup-languages" are provided for developing 
Web pages on the server site, and which are interpreted by the tag handlers 20, 
22, 24 of the system shown in Figure I . These languages are referred to herein 
as the hv-per-graphic mark-up language (HGML), the hyper-speech mark-up 
language (HSML) and hyper- video mark-up language (HVML). 
IS Designed to extend the power of the Web to mobile communications 

applications, HGML, HSML and HVML provide for fast communications over 
narrowband radio channels, such as GSM, while also being functional on higher 
bandwidth systems. 

Through the use of graphical files, audio clips and/or video clips 
2 0 provided on the mobile client terminal in a HGML^SMLmVML Toolkit, the 
need for network programming may be significantly reduced, and it is possible 
to embed tags referencing advanced graphical, speech, video and animation 
which is stored on the client side into Web pages. 

The content handler interprets the HGML/'HSML/l-iVML commands 
25 into graphics primitives and calls to access stored graphical images and 
speech/video files. It is muhi-threadedto allow multiple paints onto the page at 
a given time. 

HGML, HSML and'or HVML commands may be embedded in a Web 
page containing BITML, in which case the client side content handlers process 
30 the page and filter out the HGML, HSML and'or HVML commands before 
resolving them. 



9 



References to HGML> HSML and'or HVML resource files may be 
embedded into Web pages, containing HTML, as markups (e,g. <imgsn:- 
"images/testhginl">). 

AHemativeiy, HGML/HSML/HVML pages may be hyperlinked as 
separate resources in Web pages containing HTML {e.g. 
http ://www.test,conx''test .hgml} . 

Figure 2 iiiustrates the case in which the HGML/HSML,^VML 
resources are embedded as markups. Figure 2 details how content handlers (as 
exemplified by a HGML content handler) manage content of variable type from 
a server to display it on a Web page. 

The HGML content handier interprets the HGML commands into either 
graphics primitive or calls to access stored graphical images. There are a 
number of areas of fiinctionality (which may be defined in object classes) on the 
client side: 

■ Content handling: The upper object acting as a container for 

the other required funcSionalides; 

M Connection: To handle the network cotinection to the server 
and the retrieval of HGML resources from the server; 

m Command interpreter/Text processmg: Takes the HGML 
resource and processes it into the various graphics requirements (i.e. if more 
than one picture resides in the resource, etc); and 

M Painting: Takes the commands that make a specific HGML 
graphic and draw the required graphics to the appropriate place in the browser 
graphical user interface. Again, this could be primitives or stored images. A 
Paint object extends a thread class and is managed by tiie command interpreter, 
allowing multiple paints to be performed at the same time. 

HGML 

HGML provides for: 

1. Construction of graphics from a standard set of graphics 
primitives (circles, lines, etc). These are described below. 
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2. Recall placement of a standard set of iogo/clipart graphics 
from a client based store. Exampies are illustrated in Figures 3 aiid 4. 

3. Prevention of download of photo-realistic images. 

4. Yielded graphics which act as hyperiinks. 

5 HGML includes various tags or commands, which can be grouped into 



three functional categories. 


as shown below. 
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Drawing Primitives 

The first group of document-independent tags provide HGML with 
10 basic drawing functionality, enabling the creation of a range of simple but 
effective composite images. 

Most commands include optional attributes for (out)line colour, st>ie, 
thickness and, when appropriate, fill colour. If any of those that are not 
explicitly specified, the current default settings (as specified by the SETSTYLE 
15 tag) are used. 
ARC 
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Draws an arc of a specified width, height and angk from absolute 
starting coordinates. It has the Ibllowing attributes: 

origin X, y coordinates of arc (absolute) followed by width, height, start 
angle and arc angle, line colour, line st>'le, and line thickness, 
5 e,g. <arccoords^"20,20,50J0,45M'\color'^red> 

ARROW 

Draws a line from xLyl to x2,y2 using absolute screen (vtindow) 
coordinates, with an arrow head at the x2,y2 position. It has the following 
attributes: 

10 start and end x,y coordinates (absolute), line color, line style, and line 

thickness. 

e.g. <arrow coords^"10, lOJOO, ! 00" color ==black.psize'='"2"> 
ELLIPSE 

Draws an ellipse of a given width and height from the specified 
15 coordinates. This tag is also used to draw circles. The following attributes are 

provided: 

centre K,y coordinates (absolute) followed by width and height, outline 
colour, fill colour, outline style, and outline thickness. 

e.g. <ellipsecoorcis='"200J00,100J0".color^biackfiil''yellow> 

20 LINE 

Draws a line from xl,yl to x2,y2 using absolute screen (window) 
coordinates. Tlie attributes are: 

start and end x,y coordinates (absolute), line colour, tine style, and line 

thickness. 

2 5 e.g. <nns caords^"Hh 10. 10(1 /00'>/>'/e= "dotted". ps!ze="2"> 

LINETO 

Draws a line torn the current graphics cursor position to a new position 
(using absolute coordinates). The following attributes are defined: 

end of line x,y coordinates (absolute), line colour, line style, and line 
30 thickness, 

e.g. <limio coords- "200, 150 ", color~green> 
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LINEREL 

Draws a line relative to the current graphics cursor position using 
coordinate offsets. The following are its attributes: 

end of line coordinates (relative), line coiour, line style, atid line 
thickness. 

e.g. < linerel coords- "1 5,- 10 ". psize ~ "4 "> 

LINK 

Allows a rectangular portion of the screen to be defined as an hyperlink. 
The following attributes are given: 

rectangular region x 1 ,y 1 ,x2,y2 (absolute), hyperlink reference. 

e.g, <link caords="l 5. 10,50,60", hre/~"hup://www.demo.com/ 
demo.hgm"> 
ORIGIN 

Sets the position of the "graphics cursor". For use with LINETO and 
LINEREL tags. New graphics cursor position x.y is atiributed. 

e.g. <origin coords-" 100, 130"> 
POLYGON 

Draws a polygon using a list of absolute screen coordinates, starting 
with an origin. Coordinates are listed in the format "xl,yKx2,y2,x3,y3 
xn,yn". The line from the last specified coordinates back to the origin of the 
shape is drawn automaticaliy to comptete the polygon. The following attributes 
are provided: 

origin of shape, followed by n other points (all absolute coordinates), 
outline colour, fill colour, outline style, and outline thickness. 

e.g. <-siX'Sided-shape-> 

<polygon coords^ "100,100. 120,80. 140,80, 160. 1 00, 140, 120, 
120,120", color'=red,fill^red> 
RECTANGLE 

Draws a rectangle of a given width and height from the specified 
coordinates. This tag is also used to draw squares. The following attributes are 
defined: 
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top left x,y coordinates (absolute) foliowed by width and height 
dimensions, outline colour, fill colour, outline style, aiid outiiae thickness. 

e.g. <rectangls coords="lO, 1 0, 1 00, }S",styie = "dotied'*> 
SETSTYLE 

Enables default settings for outline colour, fill colour and outlines style 
and thickness to be specified. These settings are then used unless explicitly 
overridden by lite attributes in other HGML commands. The attributes are: 

outline colour, fill colour, outline style, outline thickness. 

e.g. <setstyk color-red, fiU'=green> 

TEXT 

Displays a text string, starting from the specified coordinates. It has the 
following attributes: 

text position x,y coordinates (absolute), text string to display, and text 

colour. 

e.g. <text coords'-"! 0,50". text^"This is an example of HGML 

tex{"> 

STORED IMAGES 

A standard range of pre-dmwn images are stored on the mobile client 
terminal, associated with a standard document-independentset of graphics tags 
as exemplified in Fig, 3 . These images may be used to provide further graphical 
content to a page, in addition to that which may be achieved using the drawing 
primitives. 
IMAGE 

Enables an image to be retrieved from the local repository and displayed 
at the specified coordinates. The current theme will be used unless another one 
is explicitly specified. A scaling factor may be specified to dynamically resize 
the image and a hyperlink can be optionally provided. The attributes are: 

image name, theme style, top left coordinates of image, scaling factor 
(percent), hyperlink reference. 

e.g. <image name-ARROWL coords=^"S0,50", scak'=^150, 
hre/== demo.coni/demo.hgm "> 
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THEME 

Enables a new current image theme to be selected. The attributes are: 

name of image theme, 

e.g <theme^DEFAULT> 

An example of a set of images which may be associated with an 
alternative theme ("Theme 2"). As shown, different themes may not have 
defined images for ali of the predefined set of tags - some may not be used in 
associated with a particular theme. Only a set number of theme styles are 
provided for. 

At a high Icvei, themes are split into three general categories: 
M Navigation 

Images of buttons for a range of relatively standard Web page functions. 
The structure of this category is strictly defined by HGML such that images are 
referenced by standard names and relate to the same type of image irrespective 
of the theme being used (i.e. HOME, SEARCH, ARROW etc). 

m Design 

A range of standard page design constracts in terms of lines, bullets, 
separators, etc. The structure of this category is strictly defmed by HGML such 
that images are referenced by standard names and relate to the same type of 
image irrespective of the theme being used (i.e. LINEU LINE2, BULLETI etc). 

The navigation and design categories have a prescribed content (i.e. the 
first element m tlie Design category is always a line etc). As such, changing the 
theme associated with a page will mean that, whilst it looks different, the 
images displayed wili stil! be meaningful and appropriate (e.g. a line will stili 
appear where intended albeit in a different style). 

m User 

User-defined images of which the content is specific to the theme. 
HGML defines standard names (i.e. USERl, USER2 etc), but these do not 
provide any indication of the type of image being referenced. 

The client side HGML functionaiits' provides at least a default theme. 
Other themes (defined for example by individual site designers) may be 
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downioaded from a remote server the first time a service that uses them is 
accessed. Aitematively, a user may have a personaJ preferred theme which may 
be provided on the client terminal and used in preference to the default and/or 
downloaded theme. 

5 User-defined theme elements are provided to permit a degree of 

flexibility to Web site developers, whilst stiil remaining optimisation 
advanages. 

This group allows local storage of other images, outside the standard 
categories prescribed. These may be used to reference any images that the 
10 site/theme designer wishes. It is anticipated that this will include images 
relating to corporate identity (e.g. logos etc) to further standardise the look and 
feel of their pages. They could, however, also include additional line and bullet 
styles etc tkit could not be encompassed under the standard headers provided. 

The number of user-defmed images is Urailed to a predetertnined 
15 number (e.g. ten) in order to prevent the image repository from becoming 
undesirably large. 

All images not included in the standard theme may need to be 
downloaded each time a site is accessed (although they may be cached from a 
previous session). At such, h is intended that user-defined slots are utilised to 
20 hold the largest and most tised images (thus helping to further optimise 
download times). 

U'Tiilst user-defmed images are envisaged primariiy as site-specific, 
however other designers may wish to reuse them in their o wn sites. In this case, 
there are two alternatives: 
25 (1) to incorporate the required images into the user-defmed slots of 

the site's own theme; or 

(2) to make the site reliant on two or more themes (i.e. a default, 
along with tlie one(s) containing the unages to be reused). In this case, the end- 
user obtains all of the necessary themes before being able to access the site. The 
3 0 site provider would provide download access for each theme. 
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Ipiage Manipulation 

HGML supports the manipulation of the onscreen bitmap image using 
COPY, PASTE, FLIP and ROTATE commands. 

These are intended to further optimise the creation of certain types of 
5 image. For example, if a composite image created using several dravtdng 
primitives is also required at another onscreen location, then it may be copied 
and pasted rather than redrawn. The orientation of it may then be changed using 
the FLIP and ROTATE operations. 

The fiinctions also help to optimise the stomge of theme images. For 
10 example, rather than store four Arrow symbols (pointing left, right, up and 
down), a single image may be stored in the database and then ROTATEd and 
FLfPed as desired. 
COPY 

Copies of specified rectangular screen segment to a memory buffer. The 
1 5 original image remains unchanged. The attributes are: 
top left and bottom right coordinates of segment. 
e.g. <copy coords= "50, 50. 1 00. 100"> 

FLIP 

Performs a horizontal or vertical flip of a screen defined by the specified 
20 coordinates. The attributes are: 

top left and bottom right coordinates of segment, and horizontal or 

vertical flip. 

e.g. <Jlip coords=^''50J0J0(U0Q'',axis''-''hor!z''> 
PASTE 

25 Copies of the contents of a memor>' buffer to an appropriately sized 

rectangular screen segment, starting at the specified coordinates. The attributes 
are: 

top left coordinates to paste segment, 
e.g. <paste caords^"lOOJOO"> 
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ROTATE 

Performs a rotation of a circular screen area by a specified angle. The 
attributes are; 

coordinates of centre, radius and rotation angle, 
e.g. <rotate coords^ "50, 50, 30", angie="80"> 
HSML 

Figure 5 illustrates a screen display on the mobile client terminal 2 when 
displaying a Web page containing HSML. 

As illustrated, the browser displays the text content, formatted as 
specified by the HTML markups in the Web page, and a speech agent image 
portion 102, which is driven in accordance with the HSML markups contained 
or referenced in the downloaded Web page in association with the replay of 
locally-stored audio clips referenced by audio tags contained in the Web page, 
or the generation of speech by conversion from text contained in the Web page. 

The speech agent interface provided by the mobile client tenninal is 
controlled by a speech playback/generaiorprogramme application and a moving 
image application programme stored on the mobile terminal . 

The speech agent has a variety of selectable vocal and facial styles. The 
various selectable vocal styles are defined in vocal style files stored on the 
mobile client terminal, containing data describing algorithms and settings for 
manipulating the audio data providing the speech function to alter the vocai 
characteristics. The facial style selections are defined in facial style files stored 
on the mobile client terminal. The facial style files include data defining a 
plurality of selectable base tacia! images, a phirality of selectable mouth images, 
a plurality of image animation algorithms (e.g. talk, smile, wink, etc.) 
corresponding with the base facial images and the mouth images, and a plurality 
of image rendering algorithms for rendering the facial style in accordance with 
selected characteristics of the face, such as skin colour, hair colour, etc. The 
vocal and facial style files may be stored permanently on the mobile terminal 
and/or stored in non-volatile memory on the mobile terminal after being 
downloaded from an Internet server. 
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The speech agent function provided in the mobile cUent terminal is 
provided with predefined default settings, in which the vocal and facial style 
characteristics are preset without input from the user. These characteristics 
include voice gender, accent, language, base facial style, skin colour, hair 
colour, eye colour, mouth size, etc. The user may however override the default 
vocal and facial style characteristics settings, by resetting one or more of the 
characteristics via menu options on a in;iii-inachine interface of the mobile 
terminal, in order to provide a personalised default speech agent. The default 
speech agent settings, whether predefined or user-defined, are used unless an 
alternative agent thenve is specified by mark-ups contained in a Web page, as 
described beiow. If an alternative agent theme is specified, the vocal style and 
the facial style of the speech agent is altered only for the duration of the display 
of the Web page in question, after which time the default speech agent is 
reverted to. 



HSML includes the following markups; 



.Audio Ptayback 


Agent Style 


Agent Control 


SAY 


AGENT THEME 


ACTION 



SAY 

This markup causes an audio file to be retrieved from, or generated 
using data retrieved from, the local repository and "spoken" by a speech agent. 
The default speech agent theme, defining the vocai and facial styles of the 
speech agent, will be used unless another one is explicitly specified, llie 
attribute required is a phrase name: 

e.g. <Aa,v phrme-'"GREETING! "> 

"GREETING!" may be a tag to an audio file held in the audio graphics 
library 28, or may be replayed using a text-to-speech translator on the client 
side. 
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AGENT THEME 

This markup causes the vocal and facial style combination for the 
speech agent to be altered from the default. Voice style attributes may be used 
in order to incorporate different genders, accents and/br languages into the 
5 piayback. Face style attributes allow different appearancesto be selected for the 
speech agent. The combination of attributes may be used to create agents with 
different personalities, Tlie attributes which may be included are: 

voice gender, accent, language, base facial style, skin colour, hair 
colour, eye colour, mouth size, etc. 
10 e.g. <agent theme~female3,EngliskEngliskfemcde5,pale,black 

brown,medium> 
ACTION 

This markup controls the image of the speech agent, determining 
whether it is to be displayed and any movements to be made in conjunction with 

15 audio playback. T!ie attributes are an action for the agent (possible actions: 
appear, talk, smile, wink, etc.) md an action activation status (on/off). e.g. 
<acdon~iaIkon> 

The action markups are referenced by a moving image handler to 
produce and control the image of the speech agent. When the appear action is 

20 acti vated, the speech agent appears at a specified part of the screen display. 

\Vhen deactivated, the speech agent does not appear. When the talk action is 
activated, the moving image handler animates the mouth of the speech agent in 
conjunction with any speech being generated by the SAY command. Other 
actions (e.g. smile, wink) are also suitably animated when activated. 

25 HVML 

HVML supports a single tag, namely PLAYBACK, which has a file 
reference attribute. When parsing Web page, the HVML tag is interpreted by 
the video tag handler, which retrieves a video file from the standard video file 
}ibrar>' 30 which corresponds with the reference contained in the command. 

30 It will be appreciated that the HGML, HSML and HVML fiinctionality 

illustrated above are provided for exemplary purposes, and are not intended to 
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be iimitiag. It wiU be appreciated that various modifjcationsand variations may 
be employed without departing from the scope of the present invention. 
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CLAIMS: 

1 . A muhimedia client terminal, said terminal comprising: 

a browser for interpreting a muitimedia document received from a 
5 remote server, said interpreting means comprising: 

means for recognising textual presentation markup tags in said 
document and presenting text to a user in accordance with said markup tags; 

means for recognising a st^-ndard set of document-independent locai 
library file markup lags in said document; 
1 0 means for storing a set of non-textual local librarv' files corresponding to 

said local library tags; and 

means for presenting the contents of one of said local library files to a 
user in response to the recognition of one of said local library file tags in said 
multimedia document 

15 

2. A multimedia document server, said server comprising storage 
means holding a set of muhimedia documents, one or more of said documents 
comprising: 

textual presentation commands; and 
20 one or more of a standard set of document-independent markup tags to 

be recognised by a client terminal downloading said one or more documents as 
references to non-textual files which are stored in said client terminal. 

3. A document retrieval system comprising a client terminal in 

25 accordance with claim i and a document server in accordance with claim 2. 

4. Apparatus according to any preceding claim, wherein said client 
terminal is a mobile communications terminal. 
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5. Apparatus according to any preceding claim, wherein said non- 
textual files comprise one or more of graphics fifes, audio files and'or video 
flies. 

5 6. Apparatus according to any preceding claim, wherein said non- 

texitial flies comprise data for speech synthesis, and said tags define speech to 

be synthesized, 

7. A communications terminal for accessing Web pages via an 
1 0 Internet protocol connection, said terminal comprising a browser for interpreting 

Web pages and presenting the contents thereof on a display portion of the 
communications terminal, said browser comprising means for recognising two 
dimensional graphics markup tags in a Web page and for rendering a graphical 
object on said display portion on the basis of a plurality of said graphics markup 
15 tags. 

8. A communications terminal according to claim 7, wherein said 
graphics markup tags comprise line drawing commands and'or image 
manipulation commands. 

20 

9. A Web server comprising Web pages describing a two 
dimensional graphical object by means of the graphics markup tags used by the 
mobile communications terminal of claim 7 or 8. 

25 10, A multimedia client terminal, said terminal comprising: 

a browser for interpreting a multimedia document received from a 
remote server, said interpreting means comprising: 

means for recognising speech presentation markup tags in said 
document; 

30 means for storing a set of local library fi les corresponding to said speech 

presentation markup tags; and 
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means for presenting said speech to a user in accordance with the iocaJ 
library files tags found in said multimedia document. 

11 . A client terminal according to claim 1 0, wherein said speech is 
5 presented in conjunction with a graphical display comprising a moving facial 

image. 

12. A client terminal according to claim 11, wherein the 
characteristics of said moving facial image are selectable by a user of the client 

10 terminal. 

13. A mobile communications terminal adapted to generate 
graphical image data defining a facial image which is animated to simulate 
actions accompanying speech generated from data received via a radio interface, 

1 5 wherem the characteristics of said facial image and/or said speech are selectable 
by a user of said cliem terminal. 



wo 99/05613 



PCT/GB&8/02177 




SUBSTITUTE SHEET (RULE 26) 



wo 99/05613 



PCT/GB9Mt2177 



2/6 




H 




EC 


u_ 


m 


O 


o 





SUBSTITUTE SHEET (RULE 26) 



wo 99/05613 



FCT/GB98/02177 



3/6 



NAVIGATION 



GRAPHICS TAG 

HOME 
BACK 

NEXT 
ARROW 1 
ARROW 2 
SEARCH 



LINKS 
FEEDBACK 



GRAPHICS TAG 
CORNER 



IMAGE 

S3 



51 



(UNDEFiNED) 



IMAGE 



UNE1 
LINE 2 
LINES 
LINE 4 

SEPERATOR 1 



SEPERATOR 2 



BULLET 1 

BULLET 2 



GRAPHICS TAG 
USER 1 



USER 



IMAGE 




FIG, 3a 



SUBSTiTUTE SHEET {RULE 26) 



wo 99/05613 



fCT/GB98/02177 



4/6 



USER (CONT.) 



GRAPHICS TAG 
USER 2 



USERS 



IMAGE 




0 



Q 



USER? 
USERS 
USER 9 
USER 10 




(UNDEFINED) 



FIG. 3b 



SUBSTITUTE SHEET (RULE 2S) 



WO99/0S613 



PCT/GB98^02177 



GRAPHiCS TAG 
HOME 

BACK 

NEXT 

ARROW 1 
ARROW 2 
SEARCH 

HELP 

LINKS 
FEEDBACK 



5/6 



NAVIGATION 



in 



DESIGN 

GRAPHiCS TAG iMAGE 



UNE 1 ' • ■ 

LINE 2 {IliniltlinillliiMlllllllilililliillllliililliliil 

LINES HMMHMHHHIHMfMHMHMHHmUHmm* 

UNE 4 1 11! I mi l II i iiiiiiii iiiiiiiiiiiii iiiiiiifi 

SEPARATOR 1 



SEPARATOR 2 




BULLET 1 
BULLET 2 



FIG. 4a 

SUBSTITUTE SHEET (RULE 2S) 



wo 09/05613 



PCT/GB98/02177 



GRAPHICS TAG 
USER 1 
USER 2 
USERS 
USER 4 
USERS 
USERS 
USER 7 
USERS 
USER 9 
USER 10 



6/6 



USER 



FIG. 4b 




SUBSTITUTE SHEET (RULE 26) 



INTERNATIONAL SEARCH REPORT 



PCT/QB 98/02177 



Rdsxanf to stare m. 



EP 0 753 835 A (SONY CORP) 15 January 1997 
see page 19, line 52 - page 19, line 56; 
figures 31,34-41 



□ 



05/11/1998 



INTERNATIONAL SEARCH REPORT 



PCT/GB 98/02177 



CA 
OP 



2180899 A 
9081782 A 



13-01-1997 
28-03-1997 



Fnrm FCTKSAffil 9 Ifsaient tsroiiy aantx) (JUy sSSSi 



